Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable using an engine endpoint for eth1 deposit/merge tracker #3949

Merged
merged 6 commits into from
Apr 30, 2022
Merged

Enable using an engine endpoint for eth1 deposit/merge tracker #3949

merged 6 commits into from
Apr 30, 2022

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Apr 23, 2022 •
edited
Loading

As per kiln spec v2.1 the engine endpoint also exposes eth_ methods sufficient for merge and deposit tracker. However they can be jwt auth protected.
This PR enables eth1 trackers to use the jwt auth protected engine endpoints, which would simplify specifying the post merge setups where an execution engine would need to be made available.

However if explicitly provided as an arg, it would continue to behave in the original way assuming an unprotected eth endpoint.

TODOs:

  • update the merge-interop tests to now use execution endpoints
    • geth
    • nethermind : Update awaited from nethermind team for the error of genesis not getting loaded
  • Add a cli unit test for the same

@codecov
Copy link

codecov bot commented Apr 23, 2022 •
edited
Loading

Codecov Report

Merging #3949 (32f083f) into master (97f6012) will decrease coverage by 0.03%.
The diff coverage is n/a.

❗ Current head 32f083f differs from pull request most recent head 440ce83. Consider uploading reports for the commit 440ce83 to get more accurate results

@@            Coverage Diff             @@
##           master    #3949      +/-   ##
==========================================
- Coverage   36.33%   36.30%   -0.04%     
==========================================
  Files         324      324              
  Lines        9063     9107      +44     
  Branches     1455     1468      +13     
==========================================
+ Hits         3293     3306      +13     
- Misses       5597     5628      +31     
  Partials      173      173

@g11tech g11tech mentioned this pull request Apr 23, 2022
Closed
22 tasks
@github-actions
Copy link
Contributor

github-actions bot commented Apr 23, 2022 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 32f083f Previous: c754818 Ratio
BeaconState.hashTreeRoot - No change 437.00 ns/op 426.00 ns/op 1.03
BeaconState.hashTreeRoot - 1 full validator 57.278 us/op 56.264 us/op 1.02
BeaconState.hashTreeRoot - 32 full validator 538.51 us/op 541.13 us/op 1.00
BeaconState.hashTreeRoot - 512 full validator 6.0438 ms/op 6.0793 ms/op 0.99
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 67.933 us/op 69.606 us/op 0.98
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 952.66 us/op 956.19 us/op 1.00
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 13.736 ms/op 12.384 ms/op 1.11
BeaconState.hashTreeRoot - 1 balances 53.556 us/op 46.709 us/op 1.15
BeaconState.hashTreeRoot - 32 balances 462.27 us/op 408.99 us/op 1.13
BeaconState.hashTreeRoot - 512 balances 4.5604 ms/op 4.0810 ms/op 1.12
BeaconState.hashTreeRoot - 250000 balances 96.322 ms/op 93.550 ms/op 1.03
processSlot - 1 slots 9.5550 us/op 8.5680 us/op 1.12
processSlot - 32 slots 1.5770 ms/op 1.4167 ms/op 1.11
getCommitteeAssignments - req 1 vs - 250000 vc 5.2688 ms/op 4.6619 ms/op 1.13
getCommitteeAssignments - req 100 vs - 250000 vc 7.3483 ms/op 6.4662 ms/op 1.14
getCommitteeAssignments - req 1000 vs - 250000 vc 7.7990 ms/op 6.9347 ms/op 1.12
computeProposers - vc 250000 16.689 ms/op 14.993 ms/op 1.11
computeEpochShuffling - vc 250000 158.79 ms/op 141.35 ms/op 1.12
getNextSyncCommittee - vc 250000 280.24 ms/op 248.33 ms/op 1.13
altair processAttestation - 250000 vs - 7PWei normalcase 4.1732 ms/op 3.6153 ms/op 1.15
altair processAttestation - 250000 vs - 7PWei worstcase 5.8406 ms/op 5.4589 ms/op 1.07
altair processAttestation - setStatus - 1/6 committees join 194.22 us/op 190.50 us/op 1.02
altair processAttestation - setStatus - 1/3 committees join 375.36 us/op 329.19 us/op 1.14
altair processAttestation - setStatus - 1/2 committees join 519.02 us/op 513.95 us/op 1.01
altair processAttestation - setStatus - 2/3 committees join 673.06 us/op 589.48 us/op 1.14
altair processAttestation - setStatus - 4/5 committees join 936.38 us/op 825.15 us/op 1.13
altair processAttestation - setStatus - 100% committees join 1.1087 ms/op 962.80 us/op 1.15
altair processBlock - 250000 vs - 7PWei normalcase 25.886 ms/op 25.297 ms/op 1.02
altair processBlock - 250000 vs - 7PWei normalcase hashState 36.754 ms/op 33.674 ms/op 1.09
altair processBlock - 250000 vs - 7PWei worstcase 72.794 ms/op 68.419 ms/op 1.06
altair processBlock - 250000 vs - 7PWei worstcase hashState 110.52 ms/op 102.95 ms/op 1.07
altair processEth1Data - 250000 vs - 7PWei normalcase 865.03 us/op 756.06 us/op 1.14
altair processEpoch - mainnet_e81889 561.07 ms/op 530.20 ms/op 1.06
mainnet_e81889 - altair beforeProcessEpoch 155.32 ms/op 141.46 ms/op 1.10
mainnet_e81889 - altair processJustificationAndFinalization 23.229 us/op 28.535 us/op 0.81
mainnet_e81889 - altair processInactivityUpdates 12.419 ms/op 12.395 ms/op 1.00
mainnet_e81889 - altair processRewardsAndPenalties 82.777 ms/op 81.833 ms/op 1.01
mainnet_e81889 - altair processRegistryUpdates 3.8270 us/op 5.3030 us/op 0.72
mainnet_e81889 - altair processSlashings 991.00 ns/op 1.8060 us/op 0.55
mainnet_e81889 - altair processEth1DataReset 1.1820 us/op 2.0800 us/op 0.57
mainnet_e81889 - altair processEffectiveBalanceUpdates 6.3005 ms/op 6.0741 ms/op 1.04
mainnet_e81889 - altair processSlashingsReset 5.5130 us/op 7.7290 us/op 0.71
mainnet_e81889 - altair processRandaoMixesReset 5.0890 us/op 8.9730 us/op 0.57
mainnet_e81889 - altair processHistoricalRootsUpdate 1.3470 us/op 2.2130 us/op 0.61
mainnet_e81889 - altair processParticipationFlagUpdates 3.0960 us/op 4.4220 us/op 0.70
mainnet_e81889 - altair processSyncCommitteeUpdates 956.00 ns/op 3.7190 us/op 0.26
mainnet_e81889 - altair afterProcessEpoch 184.69 ms/op 196.45 ms/op 0.94
altair processInactivityUpdates - 250000 normalcase 36.632 ms/op 34.590 ms/op 1.06
altair processInactivityUpdates - 250000 worstcase 30.916 ms/op 26.212 ms/op 1.18
altair processRewardsAndPenalties - 250000 normalcase 84.900 ms/op 114.81 ms/op 0.74
altair processRewardsAndPenalties - 250000 worstcase 83.225 ms/op 105.40 ms/op 0.79
altair processSyncCommitteeUpdates - 250000 290.70 ms/op 263.75 ms/op 1.10
Tree 40 250000 create 780.55 ms/op 671.47 ms/op 1.16
Tree 40 250000 get(125000) 290.02 ns/op 254.05 ns/op 1.14
Tree 40 250000 set(125000) 2.4905 us/op 2.2402 us/op 1.11
Tree 40 250000 toArray() 33.250 ms/op 30.613 ms/op 1.09
Tree 40 250000 iterate all - toArray() + loop 33.309 ms/op 30.677 ms/op 1.09
Tree 40 250000 iterate all - get(i) 112.63 ms/op 101.08 ms/op 1.11
MutableVector 250000 create 17.988 ms/op 15.355 ms/op 1.17
MutableVector 250000 get(125000) 14.808 ns/op 11.562 ns/op 1.28
MutableVector 250000 set(125000) 644.57 ns/op 641.02 ns/op 1.01
MutableVector 250000 toArray() 6.9323 ms/op 6.4338 ms/op 1.08
MutableVector 250000 iterate all - toArray() + loop 7.1568 ms/op 6.6758 ms/op 1.07
MutableVector 250000 iterate all - get(i) 3.4381 ms/op 3.0339 ms/op 1.13
Array 250000 create 6.3980 ms/op 6.3256 ms/op 1.01
Array 250000 clone - spread 2.8057 ms/op 2.6076 ms/op 1.08
Array 250000 get(125000) 1.2250 ns/op 1.1440 ns/op 1.07
Array 250000 set(125000) 1.2710 ns/op 1.1620 ns/op 1.09
Array 250000 iterate all - loop 167.98 us/op 150.38 us/op 1.12
effectiveBalanceIncrements clone Uint8Array 300000 91.307 us/op 85.029 us/op 1.07
effectiveBalanceIncrements clone MutableVector 300000 827.00 ns/op 510.00 ns/op 1.62
effectiveBalanceIncrements rw all Uint8Array 300000 301.82 us/op 266.25 us/op 1.13
effectiveBalanceIncrements rw all MutableVector 300000 208.61 ms/op 160.73 ms/op 1.30
aggregationBits - 2048 els - zipIndexesInBitList 35.099 us/op 26.350 us/op 1.33
regular array get 100000 times 67.410 us/op 59.480 us/op 1.13
wrappedArray get 100000 times 67.475 us/op 59.482 us/op 1.13
arrayWithProxy get 100000 times 28.968 ms/op 33.157 ms/op 0.87
ssz.Root.equals 487.00 ns/op 393.00 ns/op 1.24
byteArrayEquals 470.00 ns/op 382.00 ns/op 1.23
phase0 processBlock - 250000 vs - 7PWei normalcase 3.9326 ms/op 3.5434 ms/op 1.11
phase0 processBlock - 250000 vs - 7PWei worstcase 46.836 ms/op 41.033 ms/op 1.14
phase0 afterProcessEpoch - 250000 vs - 7PWei 176.96 ms/op 157.84 ms/op 1.12
phase0 beforeProcessEpoch - 250000 vs - 7PWei 77.288 ms/op 74.871 ms/op 1.03
phase0 processEpoch - mainnet_e58758 523.01 ms/op 481.96 ms/op 1.09
mainnet_e58758 - phase0 beforeProcessEpoch 237.58 ms/op 227.83 ms/op 1.04
mainnet_e58758 - phase0 processJustificationAndFinalization 25.419 us/op 27.934 us/op 0.91
mainnet_e58758 - phase0 processRewardsAndPenalties 134.14 ms/op 128.66 ms/op 1.04
mainnet_e58758 - phase0 processRegistryUpdates 9.8880 us/op 12.021 us/op 0.82
mainnet_e58758 - phase0 processSlashings 1.0870 us/op 2.0100 us/op 0.54
mainnet_e58758 - phase0 processEth1DataReset 1.1310 us/op 1.5170 us/op 0.75
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 6.5963 ms/op 5.8784 ms/op 1.12
mainnet_e58758 - phase0 processSlashingsReset 5.1600 us/op 6.8110 us/op 0.76
mainnet_e58758 - phase0 processRandaoMixesReset 6.0830 us/op 7.8370 us/op 0.78
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.2840 us/op 2.1170 us/op 0.61
mainnet_e58758 - phase0 processParticipationRecordUpdates 5.3570 us/op 6.7190 us/op 0.80
mainnet_e58758 - phase0 afterProcessEpoch 153.20 ms/op 142.11 ms/op 1.08
phase0 processEffectiveBalanceUpdates - 250000 normalcase 6.4693 ms/op 5.8760 ms/op 1.10
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 6.8491 ms/op 6.2155 ms/op 1.10
phase0 processRegistryUpdates - 250000 normalcase 9.3780 us/op 8.2500 us/op 1.14
phase0 processRegistryUpdates - 250000 badcase_full_deposits 429.04 us/op 372.02 us/op 1.15
phase0 processRegistryUpdates - 250000 worstcase 0.5 222.87 ms/op 207.69 ms/op 1.07
phase0 getAttestationDeltas - 250000 normalcase 15.482 ms/op 16.042 ms/op 0.97
phase0 getAttestationDeltas - 250000 worstcase 15.778 ms/op 16.588 ms/op 0.95
phase0 processSlashings - 250000 worstcase 5.6014 ms/op 5.8520 ms/op 0.96
shuffle list - 16384 els 10.716 ms/op 9.3947 ms/op 1.14
shuffle list - 250000 els 153.95 ms/op 136.27 ms/op 1.13
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 362.35 us/op 323.82 us/op 1.12
pass gossip attestations to forkchoice per slot 3.4106 ms/op 3.5855 ms/op 0.95
computeDeltas 3.7879 ms/op 4.0784 ms/op 0.93
computeProposerBoostScoreFromBalances 503.42 us/op 443.57 us/op 1.13
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.3186 ms/op 2.2945 ms/op 1.01
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 80.154 us/op 72.186 us/op 1.11
BLS verify - blst-native 1.8616 ms/op 1.6388 ms/op 1.14
BLS verifyMultipleSignatures 3 - blst-native 3.8059 ms/op 3.3602 ms/op 1.13
BLS verifyMultipleSignatures 8 - blst-native 8.2000 ms/op 7.2293 ms/op 1.13
BLS verifyMultipleSignatures 32 - blst-native 29.706 ms/op 26.189 ms/op 1.13
BLS aggregatePubkeys 32 - blst-native 39.342 us/op 34.575 us/op 1.14
BLS aggregatePubkeys 128 - blst-native 153.73 us/op 134.99 us/op 1.14
getAttestationsForBlock 64.791 ms/op 58.818 ms/op 1.10
CheckpointStateCache - add get delete 10.706 us/op 9.9550 us/op 1.08
validate gossip signedAggregateAndProof - struct 4.2714 ms/op 3.7791 ms/op 1.13
validate gossip attestation - struct 2.0264 ms/op 1.7786 ms/op 1.14
pickEth1Vote - no votes 2.1977 ms/op 1.9835 ms/op 1.11
pickEth1Vote - max votes 20.496 ms/op 18.932 ms/op 1.08
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.135 ms/op 10.114 ms/op 1.10
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 21.002 ms/op 19.540 ms/op 1.07
pickEth1Vote - Eth1Data fastSerialize value x2048 1.6746 ms/op 1.5159 ms/op 1.10
pickEth1Vote - Eth1Data fastSerialize tree x2048 14.012 ms/op 12.946 ms/op 1.08
bytes32 toHexString 1.0070 us/op 909.00 ns/op 1.11
bytes32 Buffer.toString(hex) 729.00 ns/op 642.00 ns/op 1.14
bytes32 Buffer.toString(hex) from Uint8Array 943.00 ns/op 855.00 ns/op 1.10
bytes32 Buffer.toString(hex) + 0x 738.00 ns/op 659.00 ns/op 1.12
Object access 1 prop 0.35300 ns/op 0.31400 ns/op 1.12
Map access 1 prop 0.30200 ns/op 0.26000 ns/op 1.16
Object get x1000 17.384 ns/op 16.017 ns/op 1.09
Map get x1000 1.0780 ns/op 0.87100 ns/op 1.24
Object set x1000 108.61 ns/op 103.86 ns/op 1.05
Map set x1000 67.131 ns/op 63.336 ns/op 1.06
Return object 10000 times 0.36790 ns/op 0.32940 ns/op 1.12
Throw Error 10000 times 5.9720 us/op 5.1909 us/op 1.15
enrSubnets - fastDeserialize 64 bits 961.00 ns/op 893.00 ns/op 1.08
enrSubnets - ssz BitVector 64 bits 742.00 ns/op 698.00 ns/op 1.06
enrSubnets - fastDeserialize 4 bits 405.00 ns/op 380.00 ns/op 1.07
enrSubnets - ssz BitVector 4 bits 757.00 ns/op 695.00 ns/op 1.09
RateTracker 1000000 limit, 1 obj count per request 184.22 ns/op 166.83 ns/op 1.10
RateTracker 1000000 limit, 2 obj count per request 138.56 ns/op 126.46 ns/op 1.10
RateTracker 1000000 limit, 4 obj count per request 115.49 ns/op 106.22 ns/op 1.09
RateTracker 1000000 limit, 8 obj count per request 103.34 ns/op 92.675 ns/op 1.12
RateTracker with prune 4.3520 us/op 3.6640 us/op 1.19
array of 16000 items push then shift 3.1571 us/op 2.7340 us/op 1.15
LinkedList of 16000 items push then shift 18.729 ns/op 16.556 ns/op 1.13
array of 16000 items push then pop 208.30 ns/op 199.83 ns/op 1.04
LinkedList of 16000 items push then pop 16.497 ns/op 15.676 ns/op 1.05
array of 24000 items push then shift 4.5647 us/op 4.1448 us/op 1.10
LinkedList of 24000 items push then shift 22.739 ns/op 21.027 ns/op 1.08
array of 24000 items push then pop 197.81 ns/op 183.55 ns/op 1.08
LinkedList of 24000 items push then pop 19.204 ns/op 18.420 ns/op 1.04

by benchmarkbot/action

@g11tech g11tech marked this pull request as ready for review April 26, 2022 14:06
@g11tech g11tech requested a review from a team as a code owner April 26, 2022 14:06
@g11tech g11tech merged commit 2d58a44 into master Apr 30, 2022
@g11tech g11tech deleted the g11tech/use-executionendpoint-for-eth1 branch April 30, 2022 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@twoeths twoeths twoeths approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@g11tech @twoeths